empirical validation
Adaptive Weighted Loss for Sequential Recommendations on Sparse Domains
Mittal, Akshay, Venkatesh, Vinay, Kandi, Krishna, Sudarshan, Shalini
The effectiveness of single-model sequential recommendation architectures, while scalable, is often limited when catering to "power users" in sparse or niche domains. Our previous research, PinnerFormerLite, addressed this by using a fixed weighted loss to prioritize specific domains. However, this approach can be sub-optimal, as a single, uniform weight may not be sufficient for domains with very few interactions, where the training signal is easily diluted by the vast, generic dataset. This paper proposes a novel, data-driven approach: a Dynamic Weighted Loss function with comprehensive theoretical foundations and extensive empirical validation. We introduce an adaptive algorithm that adjusts the loss weight for each domain based on its sparsity in the training data, assigning a higher weight to sparser domains and a lower weight to denser ones. This ensures that even rare user interests contribute a meaningful gradient signal, preventing them from being overshadowed. We provide rigorous theoretical analysis including convergence proofs, complexity analysis, and bounds analysis to establish the stability and efficiency of our approach. Our comprehensive empirical validation across four diverse datasets (MovieLens, Amazon Electronics, Yelp Business, LastFM Music) with state-of-the-art baselines (SIGMA, CALRec, SparseEnNet) demonstrates that this dynamic weighting system significantly outperforms all comparison methods, particularly for sparse domains, achieving substantial lifts in key metrics like Recall at 10 and NDCG at 10 while maintaining performance on denser domains and introducing minimal computational overhead.
relevant to the NeurIPS community " (R1) and to be addressing " a well-motivated problem " (R3) in " an important area "
We thank the reviewers for their useful and thoughtful feedback. We are glad to see that our work was found " highly That is, " the experiments show that the proposed method outperforms the baselines and We address the reviewers' comments below and ' we refer to our method for noisy inference. " How does the estimation error [of Will it change the claims of the paper? Thus, our claims are unaffected. While we share the reviewers' desire for convergence guarantees, we also note that Thus, we go from Eq. 7 to Eq. 8 by swapping Re. the noiseless case, see Figure 1 In Eq. 7, the prior density is included in The ablation studies are mentioned in the text, and fully reported in the Supplement (E.3, 'Lesion study').
Empirical Validation of the Independent Chip Model
The independent chip model (ICM) forms a cornerstone of all modern poker tournament strategy. However, despite its prominence, the ICM's performance in the real world has not been sufficiently scrutinized, especially at a large scale. In this paper, we introduce our new dataset of poker tournaments, consisting of results of over ten thousand events. Then, using this dataset, we perform two experiments as part of a large-scale empirical validation of the ICM. First, we verify that the ICM performs more accurately than a baseline we propose. Second, we obtain empirical evidence of the ICM underestimating the performances of players with larger stacks while overestimating those who are short-stacked. Our contributions may be useful to future researchers developing new algorithms for estimating a player's value in poker tournaments.
Review for NeurIPS paper: High-contrast "gaudy" images improve the training of deep neural network models of visual cortex
This paper had borderline scores. Overall, I think this paper presents a nice core finding that was sufficiently well validated in the context of simulations. The simulated results are reasonably compelling and relatively thorough analyses were presented. In the discussion with reviewers, there was a reasonable consensus that the author response overemphasized the extent to which the reviewers were hung up on the lack of experimental data. While R3 felt most strongly that this specific paper was not strong enough without further empirical validation, this was clarified to not be a bias against simulation papers generally, but rather the reviewer's opinion that they were not convinced this specific paper's results were strong enough without real data.
Review for NeurIPS paper: Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics
Weaknesses: The specific empirical evaluation chosen is the primary weakness of the paper. From a neuroscience perspective, the validation of parameter recovery on synthetic data is a necessary first step, but not a sufficient one. Given that [a] the task is primarily of neuroscientific interest and [b] a simpler (though also bayesian belief-updating) fit model is given in the cited prior work, the lack of comparison of cross-validated performance against that prior model is surprising. We should either see better cross-validation performance to the models in prior work, or similar performance but more insight / explanation of the underlying mental computation. This would show us a real payoff of the new insights here.
Reviews: Covariate-Powered Empirical Bayes Estimation
This theory paper provides a number of novel results, including theoretical analysis of minimax bounds and an empirical analysis, for combinations of relatively simple statistical estimators and machine learning models of covariate information. The paper shows that these combinations improve on both the simple estimator alone and the machine learning model alone. The main concern raised by the reviewers is that the paper provides limited empirical validation. I disagree with this assessment, as the paper should be seen as a machine learning theory paper. As the proposed framework includes a number of advanced machine learning models, including XGBoost it should be very relevant for the NeurIPS community.
Reviews: Learning Stable Deep Dynamics Models
The paper presents a method for constructing neural network architectures that have build-in theoretical guarantees of Lyapunov stability - meaning that the equilibrium will be in the origin and for any initial condition, the network will produce trajectories that converge to the equilibrium. The method is evaluated on the N-link pendulum and video generation problems. The method's significance comes from two different reasons. First, Lyapunov stability for the system is very difficult to prove with classical methods. Second, deep learning methods are largely empirical, without theoretical guarantees, limiting their applicability for life-critical system.
Advancements in Robotics Process Automation: A Novel Model with Enhanced Empirical Validation and Theoretical Insights
Pandy, Gokul, Jayaram, Vivekananda, Krishnappa, Manjunatha Sughaturu, Ingole, Balaji Shesharao, Ganeeb, Koushik Kumar, Joseph, Shenson
Abstract: Robotics Process Automation (RPA) is revolutionizing business operations by significantly enhancing efficiency, productivity, and operational excellence across various industries. This manuscript delivers a comprehensive review of recent advancements in RPA technologies and proposes a novel model designed to elevate RPA capabilities. Incorporating cutting-edge artificial intelligence (AI) techniques, advanced machine learning algorithms, and strategic integration frameworks, the proposed model aims to push RPA's boundaries. The paper includes a detailed analysis of functionalities, implementation strategies, and expanded empirical validation through rigorous testing across multiple industries. Theoretical insights underpin the model's design, offering a robust framework for its application.